Rethinking Graph Regularization for Graph Neural Networks

The graph Laplacian regularization term is usually used in semi-supervised representation learning to provide structure information for a model f(X). However, with the recent popularity of neural networks (GNNs), directly encoding A into model, i.e., f(A, X), has become more common approach. While we show that brings little-to-no benefit existing GNNs, and propose simple but non-trivial variant regularization, called Propagation-regularization (P-reg), boost performance GNN models. We formal analyses P-reg not only infuses extra (that captured by traditional regularization) also capacity equivalent an infinite-depth convolutional network. demonstrate can effectively models on both node-level graph-level tasks across many different datasets.